How would you get an infinite loop anyway? This method only loops for AggregateException.InnerExceptions, and I don't see how that could ever return a collection that includes the same AggregateException:

• AggregateException.InnerExceptions returns ReadOnlyCollection<Exception> so one cannot use that to modify the InnerExceptions collection of an existing AggregateException.
• AggregateException.InnerExceptions is not virtual so a class derived from AggregateException cannot override InnerExceptions to return something circular.
• The AggregateException constructor copies the list of inner exceptions so AggregateException.InnerExceptions won't be affected if the caller later modifies the original list.
• C# does not allow the following so you cannot pass the AggregateException reference to its own constructor.

  using System;
  
  public class MaliciousException : AggregateException {
      public MaliciousException()
          : base(innerExceptions: [this]) // error CS0027: Keyword 'this' is not available in the current context
      {
      }
  }

Exponential complexity would be possible though, if there are $m$ layers of $n$ AggregateException instances and each exception in layer $k$ has all exceptions in layer $k+1$ as its inner exceptions. So the HashSet will in theory protect against that.